Transmitting device, transmitting method, receiving device, and receiving method

ABSTRACT

Display with an appropriate luminance dynamic range is realizable on a receiving side. A gamma curve is applied to input video data having a level range from 0% to 100%*N (N: a number larger than 1) to obtain transmission video data. This transmission video data is transmitted together with auxiliary information used for converting a high-luminance level on the receiving side. A high-level side level range of the transmission video data is converted on the receiving side such that a maximum level becomes a predetermined level based on the auxiliary information received together with the transmission video data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/784,353, filed Oct. 14, 2015, which is a National Stage ofPCT/JP2014/060877, filed Apr. 16, 2014, and claims the benefits ofpriority under 35 U.S.C. § 119 of Japanese Application No. 2013-096056,filed Apr. 30, 2013. The entire contents of which are incorporatedherein by reference.

TECHNICAL FIELD

The present technology relates to a transmitting device, a transmittingmethod, a receiving device, and a receiving method, and moreparticularly to a transmitting device and others for transmittingtransmission video data obtained by application of a gamma curve.

BACKGROUND ART

Virtual reality of a high-quality image is improvable by increasing asynchronous reproduction ability for synchronous reproduction of aluminance minimum level and a luminance maximum level at the time ofimage display. This synchronous reproduction ability is sometimes calleda display dynamic range.

A conventional standard has been set to a white luminance value of 100cd/m2 throughout cases from camera-imaging to monitoring display. Inaddition, a 25 conventional transmission has been set to 8-bittransmission (representable gradations: 0 to 255) as a precondition. Therepresentable gradations are expandable by the use of 10-bittransmission or larger-bit transmission, for example. Gamma correctionis further known as a correction of gamma characteristics of a displayachieved by input of data having characteristics opposite to thecharacteristics of the display.

For example, Non-Patent Document 1 describes transmission of a videostream generated by encoding transmission video data which has beenobtained by application of a gamma curve to input video data havinglevels of 0 to 100%*N (N: larger than 1), for example.

CITATION LIST Non-Patent Document

-   Non-Patent Document 1: High Efficiency Video Coding (HEVC) text    specification draft 10 (for FDIS & Last Call)

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

An object of the present technology is to realize display with anappropriate luminance dynamic range on a receiving side.

Solutions to Problem

A concept of the present technology is directed to a transmitting deviceincluding:

a processing unit that applies a gamma curve to input video data havinga level range from 0% to 100%*N (N: a number larger than 1) to obtaintransmission video 30 data; and

a transmission unit that transmits the transmission video data togetherwith auxiliary information used for converting a high-luminance level ona receiving side.

According to the present technology, the processing unit applies a gammacurve to input video data having a level range from 0% to 100%*N (N: anumber larger than 1) to obtain transmission video data. Thetransmission unit transmits the transmission video data together withauxiliary information used for converting a high-luminance level on areceiving side. For example, the transmission unit may transmit acontainer in a predetermined format that contains a video streamobtained by encoding the transmission video data. An auxiliaryinformation insertion unit that inserts the auxiliary information into alayer of the video stream and/or a layer of the container may beprovided.

For example, according to the present technology, the processing unitmay further execute a process for converting a level of output videodata obtained by applying the gamma curve to the input video data, whichlevel corresponds to a level of the input video data in a range from100% to 100%*N, into a level corresponding to 100% of the input videodata so as to obtain the transmission video data. In this case, theauxiliary information may contain information on a filter applied topixel data of the transmission video data at a level corresponding to100% of the input video data.

For example, according to the present technology, the processing unitmay further execute a process for converting a level of output videodata obtained by applying the gamma curve to the input video data, whichlevel corresponds to a level of the input video data in a range from athreshold equal to or lower than a level corresponding to 100% to100%*N, into a level in a range from the threshold to a levelcorresponding to 100% of the input video data so as to obtain thetransmission video data.

In this case, the auxiliary information may contain information on afilter applied to pixel data of the transmission video data in a rangefrom the threshold to a level corresponding to 100% of the input videodata. Alternatively, in this case, the auxiliary information 15 maycontain information on a conversion curve applied to pixel data of thetransmission video data in a range from the threshold to a levelcorresponding to 100% of the input video data.

According to the present technology, the processing unit may use outputvideo data as the transmission video data without a change, which outputvideo data is obtained by applying the gamma curve to the input videodata. In this case, the auxiliary information may contain information ona conversion curve applied to a high-level side of the transmissionvideo data.

According to the present technology, therefore, the transmission videodata obtained by applying the gamma curve to the input video data havingthe level range from 0% to 100%*N is transmitted together with theauxiliary information used for converting the high-luminance level onthe receiving side. Accordingly, the receiving side is capable ofconverting the high-luminance level of the transmission video data basedon the auxiliary information.

For example, video data with a high dynamic range is obtainable byconverting transmission video data with a low dynamic range having alevel corresponding to 100% level of the input video data as the maximumlevel such that the maximum level becomes high. In addition, video datawith a low dynamic range, for example, is obtainable by convertingtransmission video data with a high dynamic range having a levelcorresponding to 100%*N level of the input video data as the maximumlevel such that the maximum level becomes low. Accordingly, display withan appropriate luminance dynamic range is realizable on the receivingside.

For example, according to the present technology, an identificationinformation insertion unit may be provided. This identificationinformation insertion unit inserts, into the layer of the container,identification information that indicates that the auxiliary informationhas been inserted into the layer of the video stream. In this case, thereceiving side is capable of recognizing insertion of the auxiliaryinformation into this video stream without the necessity of decoding thevideo stream, and therefore appropriately extracting the auxiliaryinformation from the video stream.

Another concept of the present technology is directed to a receivingdevice including:

a reception unit that receives transmission video data obtained byapplying a gamma curve to input video data having a level range from 0%to 100%*N (N: a number larger than 1); and

a processing unit that converts a high-level side level range of thetransmission video data such that a maximum level becomes apredetermined level based on auxiliary information received togetherwith the transmission video data.

According to the present technology, the reception unit receivestransmission video data. This transmission video data is obtained byapplying a gamma curve to input video data having a level range from 0%to 100%*N (N: a number larger than 1). The processing unit converts ahigh-level side level range of the transmission video data such that amaximum level becomes a predetermined level based on auxiliaryinformation received together with the transmission video data.

For example, the processing unit may determine the predetermined levelbased on information on the N and information on a luminance dynamicrange of a monitor contained in the auxiliary information. For example,the reception unit transmits a container in a predetermined format thatcontains a video stream obtained by encoding the transmission videodata. For example, the auxiliary information is inserted into a layer ofthe video stream.

For example, according to the present technology, the transmission videodata may be video data obtained by further executing a process forconverting a level of output video data obtained by applying the gammacurve to 5 the input video data, which level corresponds to a level ofthe input video data in a range from 100% to 100%*N, into a levelcorresponding to 100% of the input video data. The processing unit mayconvert levels of respective pixel data corresponding to 100% of theinput 10 video data into levels in a range from a level corresponding to100% of the input video data to the predetermined level by applying afilter specified in filter information contained in the auxiliaryinformation.

According to the present technology, the transmission video data may bevideo data obtained by further executing a process for converting alevel of output video data obtained by applying the gamma curve to theinput video data, which level corresponds to a level of the input videodata in a range from a threshold equal to or lower than a levelcorresponding to 100% to 100%*N, into a level in a range from thethreshold to a level corresponding to 100% of the input video data. Theprocessing unit may convert levels of respective pixel data in a rangefrom the threshold to a level corresponding to 100% of the input videodata into levels in a range from the threshold to the predeterminedlevel by applying a filter specified in filter information contained inthe auxiliary information.

According to the present technology, the transmission video data may bevideo data obtained by further executing a process for converting alevel of output video data obtained by applying the gamma curve to theinput video data, which level corresponds to a level of the input videodata in a range from a threshold equal to or lower than a levelcorresponding to 100% to 100%*N, into a level in a range from thethreshold to a level corresponding to 100% of the input video data. Theprocessing unit may convert levels of respective pixel data in a rangefrom the threshold to a level corresponding to 100% of the input videodata into levels in a range from the threshold to the predeterminedlevel by applying conversion curve information contained in theauxiliary information.

According to the present technology, output video data may be used asthe transmission video data without a change, which output video data isobtained by applying the gamma curve to the input video data. Theprocessing unit may convert levels of respective pixel data of thetransmission video data in a range from a threshold equal to or lowerthan a level corresponding 100% of the input video data to a levelcorresponding to 100%*N of the input video data into levels in a rangefrom the threshold to the predetermined level corresponding to L*100%(L: a number equal to or smaller than N) of the input video data byapplying conversion curve information contained in the auxiliaryinformation.

According to the present technology, therefore, the transmission videodata obtained by applying the gamma curve to input video data having thelevel range from 0% to 100%*N is received. Then, the high-level sidelevel range of this transmission video data is converted such that themaximum level becomes the predetermined level, based on the auxiliaryinformation received together with the transmission video data.Accordingly, display with an appropriate luminance dynamic range isrealizable, for example.

Effects of the Invention

According to the present technology, display with an appropriateluminance dynamic range is realizable on the receiving side. The effectsdescribed in this specification are only presented by way of example,and not given for any purposes of limitations. Other additional effectsmay be produced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of atransmitting and receiving system according to an embodiment.

FIG. 2 is a view illustrating transmission video 25 data (a) obtained byapplying a gamma curve.

FIG. 3 is a view illustrating transmission video data (b) obtained byapplying a gamma curve.

FIG. 4 is a view illustrating transmission video data (c) obtained byapplying a gamma curve.

FIG. 5 is a view illustrating gamma curve information inserted to alayer of a video stream.

FIG. 6 is a view illustrating a conversion process executed on thereceiving side for the transmission video data (a).

FIG. 7 is a view illustrating a conversion process 5 executed on thereceiving side for the transmission video data (b).

FIGS. 8A through 8C illustrate examples of a relationship betweenluminance sample values and pixel frequencies (frequencies).

FIG. 9 is a view illustrating a conversion process executed on thereceiving side for the transmission video data (c).

FIG. 10 is a block diagram illustrating a configuration example of atransmitting device 100.

FIG. 11 is a view illustrating an access unit at a head of GOP when anencoding system is HEVC.

FIG. 12 is a view illustrating an access unit at a position other thanthe head of the GOP when the encoding system is HEVC.

FIG. 13 is a view illustrating a structure example of a tone mappinginformation SEI message.

FIG. 14 is a view illustrating the structure example of the tone mappinginformation SEI message.

FIG. 15 is a view illustrating contents of chief 25 information in thestructure example of the tone mapping information SEI message.

FIG. 16 is a view illustrating a structure example of an HDR conversionSEI message.

FIG. 17 is a view illustrating contents of chief 30 information in thestructure example of the HDR conversion SEI message.

FIG. 18 is a view illustrating a structure example of an HDR simpledescriptor.

FIG. 19 is a view illustrating contents of chief information in thestructure example of the HDR simple descriptor.

FIG. 20 is a view illustrating a structure example of an HDR fulldescriptor.

FIG. 21 is a view illustrating a structure example of a level mappingcurve descriptor.

FIG. 22 is a view illustrating a conversion curve (mapping curve) forconverting high-level side levels of transmission video data.

FIG. 23 is a view schematically illustrating an example of a mappingcurve table.

FIG. 24 is a view illustrating a configuration example of an MPEG2transport stream containing various types of SEI messages anddescriptors.

FIG. 25 is a block diagram illustrating a configuration example of areceiving device.

FIG. 26 is a block diagram illustrating a configuration example of anHDR processing unit of the receiving device.

FIGS. 27A through 27D are views illustrating processes of respectiveunits of the HDR processing unit 25 when filter information is used.

FIG. 28 is a view illustrating a process of a range mapping processingunit when the filter information is used.

FIG. 29 is a view illustrating the process of the 30 range mappingprocessing unit when the filter information is used.

FIG. 30 is a view illustrating a process of the range mapping processingunit when conversion curve information is used.

FIG. 31 is a block diagram illustrating a configuration example of anMPEG-DASH base stream distribution system.

FIGS. 32A through 32 D are views illustrating an example of arelationship between respective structures disposed in an MPD file in ahierarchical manner.

FIGS. 33A through 33C are views illustrating a structure of DASHspecification stream segments.

FIG. 34 is a view schematically illustrating information within atransport stream, which information corresponds to information containedin “Initialization Segment” and information contained in “Media Segment”in a segment data format corresponding to MPEG-2TS.

FIG. 35 is a block diagram illustrating a configuration example of atransmitting and receiving system which handles an MMT structuretransmission stream.

FIG. 36 is a view illustrating a configuration of an MMT packet in atree form.

FIG. 37 is a view illustrating a structure example of an HDR descriptionmessage having an HDR simple description table.

FIG. 38 is a view illustrating a structure example of the HDR simpledescription table.

FIG. 39 is a view illustrating a structure example of an HDR descriptionmessage having an HDR full description table.

FIG. 40 is a view illustrating a structure example of the HDR fulldescription table.

FIG. 41 is a view illustrating a structure example of an HDR descriptionmessage having a level mapping curve table.

FIG. 42 is a view illustrating a structure example 5 of the levelmapping curve table.

MODE FOR CARRYING OUT THE INVENTION

A mode for carrying out the invention (hereinafter referred to as‘embodiment”) is now described. The description is presented in thefollowing order.

1. Embodiment

2. Modified Example

1. Embodiment

[Configuration Example of Transmitting and Receiving System]

FIG. 1 illustrates a configuration example of a transmitting andreceiving system 10 according to an embodiment. The transmitting andreceiving system 10 is constituted by a transmitting device 100 and areceiving device 200.

The transmitting device 100 generates an MPEG2 transport stream TS as acontainer, and transmits the transport stream TS carried on broadcastingwaves. The transport stream TS includes a video stream obtained byencoding transmission video data to which a gamma curve has beenapplied.

According to this embodiment, the transmission video data is obtained byapplying a gamma curve to input video data with HDR (High Dynamic Range)which has been obtained by camera-imaging, i.e., input video data havinga level range from 0 to 100%*N (N: number larger than 1), for example.It is assumed herein that the 100% level is a luminance levelcorresponding to a white luminance value of 100 cd/m2.

The transmission video data includes transmission video data (a),transmission video data (b), and transmission video data (c) discussedhereinbelow, for example. The transmission video data (a) and thetransmission video data (b) have the maximum level corresponding to the100% level of input video data, and constitutes video data with a lowdynamic range. The transmission video data (c) has the maximum levelcorresponding to the 100%*N level of input video data, and constitutesvideo data with a high dynamic range.

“Transmission Video Data (a)”

The transmission video data (a) is herein described with reference toFIG. 2. In this figure, “Content data level range” indicates a levelrange from 0% to 100%*N of input video data. In this figure, “V_100*N”indicates a level of video data (output video data) corresponding to the100%*N level of input video data and obtained after application of agamma curve. In this figure, “V_100” indicates a level of video data(output video data) corresponding to the 100% level of input video dataand obtained after application of the gamma curve. In this figure,“Encoder Input Pixel data range” indicates a level range of transmissionvideo data from 0 to V_100. For example, gradations from 0 to V_100 areexpressed based on predetermined bits, such as 8 bits.

The transmission video data (a) is obtained by a clipping process (seebroken line b) which further converts levels of output video data, whichdata is obtained by applying a gamma curve (see solid line a) to inputvideo data, and corresponds to levels of input video data in the rangefrom 100% to 100%*N, into levels corresponding to 100% of the inputvideo data (V_100). The transmission video data (a) has levelscorresponding to levels of input video data in the range from 0% to100%, and constitutes video data with a low dynamic range.

“Transmission Video Data (b)”

The transmission video data (b) is herein described with reference toFIG. 3. In this figure, “Content data level range”, “V_100*N”, and“Encoder Input Pixel data range” are similar to the corresponding onesillustrated in FIG. 2. In this figure, “V_th” indicates a thresholdclipping level (Threshold clipping level) as a threshold equal to orlower than a level corresponding to the 100% level of input video data.

The transmission video data (b) is obtained by a mapping process whichfurther converts levels of output video data, which data is obtained byapplying a gamma curve (see solid line a) to input video data, and liesin a range from a threshold (V_th) equal to or lower than the levelcorresponding to 100% of the input video data to a level (V_100*N)corresponding to 100%*N of input video data, into levels in a range fromthe threshold (V th) to the level (V_100) corresponding to 100% of theinput video data. The transmission video data (b) has levelscorresponding to levels of input video data in the range from 0% to100%, and constitutes video data with a low dynamic range.

“Transmission Video Data (c)”

The transmission video data (c) is herein described with reference toFIG. 4. In this figure, “Content data level range” and “V_100*N” aresimilar to the corresponding ones illustrated in FIG. 2. In this figure,“Encoder Input Pixel data range” indicates a level range from 0 to V100*N of transmission video data. The transmission video data (c) isoutput video data obtained by applying a gamma curve (see solid line a)to input video data, and not subjected to further processing. Thetransmission video data (c) has levels corresponding to levels of inputvideo data in the range from 0% to 100%*N, and constitutes video datawith a high dynamic range.

Returning to FIG. 1, the transmitting device 100 inserts informationabout the foregoing gamma curve into a layer of a video stream. Thisinformation contains

“extended_range_white_level”,

“nominal_black_level_code_value”,

“nominal_white_level_code_value”, and

“extended_white_level_code_value”, for example, as illustrated in FIG.5.

In this information, “extended_range_white_level” indicates a percentageof an integer multiple (N times) (100%*N) when “nominal while level(nominal white level) is set to 100%. In this information,“nominal_black_level_code_value” indicates a luminance sample value fora nominal black level. When video data is encoded on the basis of 8bits, a black level is set to “16”. In this information,“nominal_white_level_code_value” indicates a luminance sample value fora nominal white level. When video data is encoded on the basis of 8bits, a white level is set to “235”, for example. In this information,“extended_white_level_code_value” indicates a luminance sample value of‘extended_range_white_level”.

Moreover, the transmitting device 100 inserts auxiliary information intothe layer of the video stream, which information is used for convertinga high-level side level range of transmission video data on the 20receiving side. This auxiliary information contains filter informationand conversion curve information, for example. The auxiliary informationwill be detailed later.

Furthermore, the transmitting device 100 inserts, into a layer of atransport stream TS, identification information indicating that thegamma curve information and the auxiliary information have been insertedinto the layer of the video stream. For example, the identificationinformation is inserted as a subordinate of a program map table (PMT:Program Map Table) contained in the transport stream TS. The presence orabsence of the gamma curve information and the auxiliary information isrecognizable based on the identification information without thenecessity of decoding the video stream. The identification informationwill be detailed later.

The receiving device 200 receives the transport stream TS transmittedwhile carried on broadcasting waves from the transmitting device 100.The transport stream TS includes a video stream containing encoded videodata. The receiving device 200 acquires video data for display bydecoding the video stream, for example.

As described above, the layer of the video stream contains insertion ofthe gamma curve information and the auxiliary information. On the otherhand, the layer of the transport stream TS contains insertion of theidentification information indicating whether or not the gamma curveinformation and the auxiliary information have been inserted. Thereceiving device 200 recognizes the presence of insertion of the gammacurve information and the auxiliary information into the layer of thevideo stream based on the identification information, and acquires thesepieces of information from the video stream for utilization of thesepieces of information for processing.

The receiving device 200 converts the high-level side level range of thevideo data after decoding (transmission video data) in such a mannerthat the maximum level becomes a predetermined level based on theauxiliary information. In this case, the receiving device 200 determinesthe predetermined level based on information about N contained in theauxiliary information, and information about a luminance dynamic rangeof a monitor, for example.

When the transmission video data is the transmission video data (a),transmission video data (b), or transmission video data (c) discussedabove, the 10 receiving device 200 executes the following conversionprocesses. These conversion processes allow display with an appropriateluminance dynamic range on the receiving side.

“Conversion Process for Transmission Video Data (a)”

The conversion process for the transmission video data (a) is hereindescribed with reference to FIG. 6. In this figure, “Decoded pixel datarange” indicates a level range of input video data (transmission videodata) from 0 to V_100. In this figure, “Display Level range” indicates alevel range of a monitor (display) from 0% luminance to 100%*Nluminance. A solid line a is a curve showing gamma characteristics ofthe monitor, as characteristics opposite to the characteristics of theforegoing gamma curve (see solid line a in FIG. 2).

The receiving device 200 converts levels of respective pixel data oftransmission video data at the level of V 100 into levels within a rangefrom V 100 to a predetermined level (V_100*N or lower) by applying afilter specified in filter information contained in the auxiliaryinformation. In this case, the levels of the pixel data at the level ofV_100 in the transmission video data prior to conversion are convertedinto such levels as to generate 100% luminance or higher in the monitor(display) as indicated by a chain line b. This video data after theconversion has the maximum level equivalent to the predetermined levelhigher than V_100, and constitutes video data with a high dynamic range.

“Conversion Process for Transmission Video Data (b)”

The conversion process for the transmission video data (b) is hereindescribed with reference to FIG. 7. In this figure, ‘Decoded pixel datarange” indicates a level range of input video data (transmission videodata) from 0 to V_100. In this figure, “Display Level range” indicates alevel range of a monitor (display) from 0% luminance to 100%*Nluminance. A solid line a is a curve showing gamma characteristics ofthe monitor, as characteristics opposite to the characteristics of theforegoing gamma curve (see solid line a in FIG. 3)

The receiving device 200 converts levels of respective pixel data oftransmission video data in the range from V_th to V_100 into levelswithin a range from V_th to the predetermined level (V_100*N or lower)by applying a filter specified in the filter information or theconversion curve information contained in the auxiliary information. Inthis case, the levels of the pixel data at the levels ranging from V_thto V_100 in the transmission video data prior to conversion areconverted into such levels as to generate 100% luminance or higher inthe monitor (display) as indicated by a chain line b. This video dataafter the conversion has the maximum level equivalent to thepredetermined level higher than V_100, and constitutes video data with ahigh dynamic range.

FIGS. 8A through 8C illustrate examples of a relationship betweenluminance sample values and pixel frequencies (frequencies). FIG. 8Aillustrates a state of input video data in the transmitting device 100,where the maximum sample value is V_N*100. FIG. 8B illustrates a stateof transmission video data (output video data) after application of agamma curve in the transmitting device 100, where the maximum samplevalue is limited to V_100. In this case, pixels of sample values withina range indicated by a broken line are affected by a mapping process,and therefore deviated from the original levels.

FIG. 8C illustrates a state after the conversion process in thereceiving device 200. In this case, pixels existing in sample valueswithin a range indicated by a broken line are pixels subjected to theconversion process (re-mapping process). This re-mapping process allowsthe levels of the respective pixels affected by the mapping process toapproach the levels prior to the mapping process. According to FIG. 8C,the maximum of the sample values is V_N*100. However, the maximum of thesample values becomes a level lower than V_N*100 depending on theluminance dynamic range of the monitor (Monitor Luminance dynamicrange).

“Conversion Process for Transmission Video Data (c)”

The conversion process for the transmission video data (b) is hereindescribed with reference to FIG. 9. In this figure, “Decoded pixel datarange” indicates a level range of input video data (transmission videodata) from 0 to V_100*N. In this figure, “Display Level range” indicatesa level range of a monitor (display) from 0% luminance to 100%*Lluminance. A solid line a is a curve showing gamma characteristics ofthe monitor, as characteristics opposite to the characteristics of theforegoing gamma curve (see solid line a in FIG. 4).

The receiving device 200 converts levels of respective pixel data oftransmission video data at the levels ranging from V_th to V_100*N intolevels within a range from V_th to a predetermined level (V_100*L) byapplying conversion curve information contained in the auxiliaryinformation. In this case, the levels of the pixel data ranging fromV_th to V_100*N in the transmission video data prior to conversion areconverted into such levels as to generate V_100*L luminance or lower inthe monitor (display) as indicated by a chain line b. This video dataafter the conversion has the maximum level equivalent to a predeterminedlevel lower than V_100*N, and constitutes video data with a low dynamicrange.

“Configuration Example of Transmitting Device”

FIG. 10 illustrates a configuration example of the transmitting device100. The transmitting device 100 includes a control unit 101, a camera102, a color space conversion unit 103, a gamma processing unit 104, avideo encoder 105, a system encoder 106, and a transmission unit 107.The control unit 101 includes a CPU (Central Processing Unit), andcontrols operations of respective units of the transmitting device 100based on a control program stored in a predetermined storage.

The camera 102 images a subject, and outputs video data with HDR (HighDynamic Range). This video data has levels in a range from 0 to 100%*N,such as 0 to 400% or 0 to 800%. In this case, a 100% level correspondsto a white luminance value of 100 cd/m2. The color space conversion unit103 converts the RGB color space of video data output from the camera102 into the YUV color space.

The gamma processing unit 104 applies a gamma curve to video data aftercolor space conversion, and performs processing for convertinghigh-luminance levels (mapping process and clipping process) asnecessary, to obtain transmission video data (see FIGS. 2 through 4).This transmission video data is expressed on the basis of 8 bits in caseof the transmission video data (a) and (b), and 9 or larger bits in caseof the transmission video data (c).

The video encoder 105 encodes conversion video data using MPEG4-AVC,MPEG2video, or HEVC (high Efficiency Video Coding), for example, toobtain encoded video data. Moreover, the video encoder 105 generates avideo stream (video elementary stream) containing this encoded videodata by using a stream formatter (not shown) provided in a subsequentstage.

At this time, the video encoder 105 inserts gamma curve information andauxiliary information into a layer of the video stream. This auxiliaryinformation is information used for converting high-luminance levels onthe receiving side, and contains filter information, conversion curveinformation and others.

The system encoder 106 generates a transport stream TS containing thevideo stream generated by the video encoder 105. The transmission unit107 transmits this transport stream TS carried on broadcasting waves orpackets on a network to the receiving device 200.

At this time, the system encoder 106 inserts, into a layer of thetransport stream TS, identification information indicating whether ornot the gamma curve information and the auxiliary information have beeninserted into the layer of the video stream. The system encoder 106further inserts conversion curve data into the layer of the transportstream TS. The system encoder 106 inserts the identification informationand the conversion curve data as a subordinate of a video elementaryloop (Video ES loop) of a program map table (PMT: Program Map Table)contained in the transport stream TS, for example.

The operation of the transmitting device 100 illustrated in FIG. 10 isnow briefly described. The RGB color space of HDR video data imaged bythe camera 102 is converted into the YUV color space by the color spaceconversion unit 103. The HDR video data after the color space conversionis supplied to the gamma processing unit 104. The gamma processing unit104 applies a gamma curve to the video data after the color spaceconversion, and performs processing for converting high-luminance levels(mapping process and clipping process) for the video data as necessaryto obtain transmission video data. This transmission video data issupplied to the video encoder 105.

The video encoder 105 encodes the transmission video data by usingMPEG4-AVC (MVC), MPEG2video, or HEVC (high Efficiency Video Coding), forexample, to obtain encoded video data. The video encoder 105 generates avideo stream (video elementary stream) containing this encoded videodata. At this time, the video encoder 105 inserts gamma curveinformation into a layer of the video stream, and further insertsauxiliary information containing filter information, conversion curveinformation and the like, as auxiliary information used for convertingthe high-luminance levels on the receiving side, into the layer of thevideo stream.

The video stream generated by the video encoder 105 is supplied to thesystem encoder 106. The system encoder 106 generates an MPEG2 transportstream TS containing the video stream. At this time, the system encoder106 inserts, into a layer of the transport stream TS, the conversioncurve data, and identification information indicating that the gammacurve information and the auxiliary information have been inserted intothe layer of the video stream. The transmission unit 107 transmits thistransport stream TS carried on broadcasting waves.

[Gamma Curve Information, Auxiliary Information, IdentificationInformation, Conversion Curve Data Structure, and TS Structure]

As described above, the gamma curve information and the auxiliaryinformation are inserted into a layer of a video stream. When theencoding system is MPEG4-AVC, or other encoding systems such as HEVC,which have similar encoding structure such as the structure of NALpackets, for example, the auxiliary information is inserted into a part“SEIs” of an access unit (AU) as an SEI message.

The gamma curve information is inserted as a tone mapping informationSEI message (Tone mapping information SEI message). The auxiliaryinformation is inserted as an HDR conversion SEI message (HDR conversionSEI message).

FIG. 11 illustrates an access unit located at a head of GOP (Group OfPictures) when the encoding system is HEVC. FIG. 12 illustrates anaccess unit located at a position of GOP (Group Of Pictures) other thanthe head thereof when the encoding system is HEVC. In case of theencoding system of HEVC, an SEI message group for decoding “Prefix_SEIs”are disposed before slices (slices) where pixel data are encoded, whilean SEI message group for display “Suffix_SEIs” are disposed after theseslices (slices).

As illustrated in FIGS. 11 and 12, the tone mapping information SEImessage (Tone mapping information SEI message) and the HDR conversionSEI message (HDR conversion SEI message) are disposed as the SEI messagegroup “Suffix_SEIs”.

FIGS. 13 and 14 illustrate structure examples (Syntax) of the “Tonemapping information SEI message”. FIG. 15 illustrates contents of chiefinformation (Semantics) in the structure examples. In these figures,“Tone mapping_cancel flag” is 1-bit flag information. In this case, “1”indicates cancellation of a previous message state of the tone mapping((Tone mapping). In addition, “0” indicates transmission of respectiveelements for refreshment of a previous state.

An 8-bit field of “coded_data_bit_depth” indicates a bit length ofencoded data, and uses 8 to 14 bits, for example. In these figures,“target_bit_depth” indicates the maximum bit length assumed as an output(output) bit length in a process performed based on the tone mappinginformation SEI message, and is allowed to use 16 bits as the maximum.

A 32-bit field of “ref_screen_luminance_white” indicates a nominal whitelevel of a reference monitor, and is expressed by the unit of “cd/m2”.In these figures, “extended_range_white_level” indicates a percentage ofan integer multiple (N times) (100%*N) when “nominal while level(nominal_white_level)” is set to 100%. In these figures,“nominal_black_level_code_value” indicates a luminance sample value fora nominal black level. When video data is encoded on the basis of 8bits, a black level is set to “16”. In these figures,“nominal_white_level_code_value” indicates a luminance sample value forthe nominal white level. When video data is encoded on the basis of 8bits, the white level is set to “235”. In this information,“extended_white_level_code_value” indicates a luminance 15 sample valueof “extended_range_white_level”.

FIG. 16 illustrates a structure example (Syntax) of the “HDR_conversionSEI message”. FIG. 17 indicates contents of chief information(Semantics) in this structure example. In these figures,“HDR_conversion_cancel_flag” is 1-bit flag information. In this case,“1” indicates cancellation of a message state of a previous HDRconversion (HDR_conversion). In addition, “0” indicates transmission ofrespective elements for refreshment of a previous state.

A 16-bit field of “threshold_clipping_level” indicates a threshold ofluminance converted into a conventional encoding range by non-lineartone mapping (tone mapping) within a range of HDR. In other words,“threshold_clipping_level” indicates V_th (see FIG. 3). An 8-bit fieldof “operator type” indicates a filter type used at the time of executionof marking (Marking) of luminance levels exceeding the V_th (thresholdclipping level). An 8-bit-filed of “range_max_percent” indicates N of100%*N.

An 8-bit field of “level mapping curve type” indicates a type of afunction for converting luminance levels exceeding the V_th (thresholdclipping level) into target luminance levels. This 8-bit field of“level_mapping_curve_type” is disposed only when“threshold_clipping_level”<“nominal_white_level_code_value” holds, i.e.,when the V_th is lower than luminance 100%.

As described above, identification information indicating that gammacurve information and auxiliary information have been inserted into alayer of a video stream is inserted as a subordinate of a videoelementary loop (Video ES loop) of a program map table (PMT) of atransport stream TS, for example.

FIG. 18 illustrates a structure example (Syntax) of an HDR simpledescriptor (HDR_simple descriptor) as identification information. FIG.19 illustrates contents of chief information (Semantics) in thisstructure example.

An 8-bit field of “HDR_simple descriptor tag” indicates a descriptortype, showing that this structure is an HDR simple descriptor. An 8-bitfield of “HDR_simple descriptor length” indicates a length (size) of thedescriptor, showing a byte count of the subsequent part as the length ofthe descriptor.

A 1-bit field of “Tonemapping_SEI_existed” is flag informationindicating whether or not tone mapping SEI information (gamma curveinformation) is present in a video layer (layer of video stream). Inthis case, “1” indicates that the tone mapping SEI information ispresent, while indicates that the tone mapping SEI information isabsent.

A 1-bit field of “HDR_conversion_SEI_existed” is flag information whichindicates whether or not HDR conversion SEI information (auxiliaryinformation) is present in the video layer (layer of video stream). Inthis case, “1” indicates that the HDR conversion SEI information ispresent, while “0” indicates that the HDR conversion SEI information isabsent.

FIG. 20 illustrates a structure example (Syntax) of an HDR fulldescriptor (HDR_full descriptor) as identification information. An 8-bitfield of “HDR_full descriptor tag” indicates a descriptor type, showingthat this structure is an HDR full descriptor. An 8-bit field of“HDR_full descriptor length” indicates a length (size) of thedescriptor, showing a byte count of the subsequent part as the length ofthe descriptor.

While not detailed herein, this HDR full descriptor further includes theforegoing tone mapping information SEI message (see FIGS. 13 and 14),and HDR conversion SEI message (see FIG. 16), as well as informationcontained in the HDR simple descriptor (see FIG. 18).

In this case, it is allowed on the receiving side to recognize not onlythe presence or absence of the tone mapping SEI information and the HDRconversion SEI information in the video layer, but also informationcontents contained therein, before decoding the video stream based onthe HDR full descriptor.

As described above, the conversion curve data is further inserted as asubordinate of the video elementary loop (Video ES loop) of the programmap table (PMT) of the transport stream TS, for example. FIG. 21illustrates a structure example (Syntax) of a level mapping curvedescriptor (level_mapping_curve descriptor) as conversion curve data.

An 8-bit field of “level mapping curve descriptor tag” indicates adescriptor type, showing that this structure is a level mapping curvedescriptor. An 8-bit field of “level_mapping_curve descriptor length”indicates a length (size) of the descriptor, showing a byte count of thesubsequent part as the length of the descriptor.

An 8-bit field of “mapping_curve_table_id” indicates an identifier (id)of a table of a mapping curve (mapping curve). This“mapping_curve_table_id” allows coexistence of a plurality of types ofuse cases (Usecase). For example, the “mapping_curve_table_id” allowsdiscrimination between conversion curves (mapping curves) used for theconversion process for each of the transmission video data (b) and thetransmission video data (c).

A 16-bit field of “number of levels N” indicates a number of levelscontained in a conversion target level range of the transmission videodata. In this case, the conversion target level range is from V_th toV_100 for the transmission video data (b) (see FIG. 7), and from V_th toV_100*N for the transmission video data (c) (see FIG. 9).

An 8-bit field of “number of curve types C” indicates a type of theconversion curve (mapping curve). This “number of curve types C” allowscoexistence of a plurality of types of conversion curves havingdifferent conversion characteristics. Possible examples of conversioncurves having different conversion characteristics include conversioncurves having different maximum levels after conversion, and conversioncurves having an identical maximum level but different intermediateconversion levels.

A 16-bit field of “curve_data” indicates values of the conversion curve(mapping curve) after conversion. FIG. 22 illustrates an example ofthree types of conversion curves (mapping curves) (a), (b), and (c). Therespective examples have the maximum level of V_100*N after conversion,and have different intermediate conversion levels. FIG. 23 schematicallyillustrates a table of mapping curves (mapping curves) corresponding tothe three types of conversion curves (mapping curves) (a), (b), and (c)illustrated in FIG. 22.

FIG. 24 illustrates a configuration example of a transport stream TS.The transport stream TS contains a PES packet “PID1: video PES1” of avideo elementary stream. Tone mapping SEI information and HDR conversionSEI information are inserted into this video elementary stream.

The transport stream TS further contains a PMT (Program Map Table) asPSI (Program Specific Information). This PSI is information describingto which programs respective elementary streams contained in thetransport stream belong. The transport stream TS further contains EIT(Event Information Table) as SI (Serviced Information) for management bythe unit of an event (program).

The PMT includes an elementary loop containing information concerningrespective elementary streams According to this configuration example,the PMT includes a video elementary loop (Video ES loop). This videoelementary loop includes information such as a stream type, and a packetidentifier (PID) associated with the one video elementary streamdescribed above, and further a descriptor describing informationconcerning this video elementary stream.

The HDR simple descriptor (HDR_simple descriptor) or the HDR fulldescriptor (HDR_full descriptor) is disposed as a subordinate of thevideo elementary loop (Video ES loop) of the PMT. As discussed above,these descriptors indicate that the tone mapping SEI information and theHDR conversion SEI information have been inserted into the video stream.Moreover, a level mapping curve descriptor (level mapping curvedescriptor) is disposed as a subordinate of the video elementary loop(Video ES loop) of the PMT.

“Configuration Example of Receiving Device”

FIG. 25 is a configuration example of the receiving device 200. Thereceiving device 200 includes a control unit 201, a reception unit 202,a system decoder 203, a video decoder 204, an HDR processing unit 205, acolor space conversion unit 206, and a display unit 207. The controlunit 201 includes a CPU (Central Processing Unit), and controlsoperations of respective units of the receiving device 200 under acontrol program stored in a predetermined storage.

The reception unit 202 receives a transport stream TS transmitted fromthe transmitting device 100 while carried on broadcasting waves. Thesystem decoder 203 extracts a video stream (elementary stream) from thistransport stream TS. The system decoder 203 further extracts theforegoing HDR simple descriptor (HDR simple descriptor) or HDR fulldescriptor (HDR full descriptor) from this transport stream TS, andtransmits the extracted descriptor to the control unit 201.

The control unit 201 is capable of recognizing whether or not tonemapping SEI information and HDR conversion SEI information have beeninserted into the 5 video stream based on the descriptor. Whenrecognizing that the SEI information is present, the control unit 203 isenabled to control the video decoder 204 such that the video decoder 204positively acquires the SEI information, for example.

The system decoder 203 extracts a level mapping curve descriptor(level_mapping_curve descriptor) from this transport stream TS, andtransmits the extracted descriptor to the control unit 201. The controlunit 201 is capable of controlling, based on a table of a mapping curve(mapping curve) contained in this descriptor, a conversion processexecuted by the HDR processing unit 205 using conversion curveinformation.

The video decoder 204 acquires baseband video data (transmission videodata) by executing a decoding process for the video stream extracted bythe system decoder 203. The video decoder 204 further extracts an SEImessage inserted into the video stream, and transmits the extracted SEImessage to the control unit 201. This SEI message contains a tonemapping information SEI message (Tone mapping information SEI message)and an HDR conversion SEI message (HDR conversion SEI message). Thecontrol unit 201 controls the decoding process and a display processbased on the SEI information.

The HDR processing unit 205 converts a high-level side level range ofthe video data obtained by the video decoder 204 (transmission videodata) based on auxiliary information such that the maximum level of thevideo data becomes a predetermined level. In this case, the HDRprocessing unit 205 executes processing corresponding to thetransmission video data (a), (b), and (c), as discussed above (see FIGS.6, 7, and 9).

The HDR processing unit 205 will be detailed later.

The color space conversion unit 206 converts the YUV color space of thevideo data obtained by the HDR processing unit 205 into the RGB colorspace. The display unit 207 displays an image based on video data afterthe color space conversion.

[Configuration Example of HDR Processing Unit]

FIG. 26 illustrates a configuration example of the HDR processing unit205. The HDR processing unit 205 includes a clipping processing unit251, a marking processing unit 252, and a range mapping processing unit253. In case of the transmission video data (a) (see FIG. 6), thetransmission video data (decoded pixel data) is input to the clippingprocessing unit 251, where a process using filter information isexecuted.

In case of the transmission video data (b) (see FIG. 7), thetransmission video data (decoded pixel data) is input to the clippingprocessing unit 251 when V_th (threshold_clipping_level)=V_100. In theclipping processing unit 251, a process using filter information isexecuted.

Concerning this transmission video data (b) (see FIG. 7), either theprocess using filter information or a 5 process using conversion curveinformation is executable when V_th (threshold_clipping_level)<V_100.When the process using the filter information is executed, thetransmission video data (decoded pixel data) is input to the clippingprocessing unit 251. When the process using the conversion curveinformation is executed, the transmission video data (decoded pixeldata) is input to the range mapping processing unit 253.

In case of the transmission video data (c) (see FIG. 9), thetransmission video data (decoded pixel data) is input to the rangemapping processing unit 253, where a process using conversion curveinformation is executed.

Initially discussed is the case of execution of the process using thefilter information. The clipping processing unit 251 extracts, as atarget for a re-mapping process, pixel data at levels equal to or higherthan a level of a threshold clipping level (Threshold clipping level)from pixel data constituting the transmission video data, using thisthreshold clipping level. In case of the transmission video data (a),the threshold clipping level (Threshold_clipping_level) becomes V_100.

For example, it is assumed that FIG. 27A shows a part of pixel dataconstituting transmission video data, where only pixel data indicated inwhite corresponds to pixel data at levels equal to or higher than thethreshold clipping level. As illustrated in FIG. 27B, the clippingprocessing unit 251 extracts pixel data indicated as a white part andcorresponding to a target of the re-mapping process. In this case, theHDR processing unit 205 outputs pixel data not corresponding to thetarget of the re-mapping process without changing values of these data.

The marking processing unit 252 performs level separation for each pixeldata corresponding to the target of the re-mapping process by executingfilter type filtering operation indicated by an operator type (Operatortype), while using pixel data around the corresponding pixel data aswell. FIG. 27C illustrates a state of level separation of the respectivepixel data corresponding to the target of the re-mapping process. FIG.27D illustrates three stages of level separation, i.e., (1) “highestlevel”, (2) “2nd highest level”, and (3) “3rd highest level”. While thestages of level separation are constituted by three stages herein foreasy understanding, a larger number of stages are established in anactual situation.

The range mapping processing unit 253 maps the values of the respectivepixel data into values corresponding to the respective stages of levelseparation, and outputs the results. The range mapping processing unit253 maps the values by using a range max percent (renge_max_percent),i.e., the value N and a monitor luminance dynamic range (MonitorLuminance dynamic range).

FIG. 28 illustrates an example of range mapping. According to thisexample shown in the figure, the range max percent (renge_max_percent)is “4”, while the monitor luminance dynamic range (Monitor Luminancedynamic range) is 400%. (1) The pixel data of “highest level” is mappedto such a value that the output luminance percentage (Output luminancepercentage) corresponding to output luminance of the display unit 207becomes 400%. (2) The pixel data of “2nd highest level” is mapped tosuch a value that the output luminance percentage becomes 300% (3) Thepixel data of “3rd highest level” is mapped to such a value that theoutput luminance percentage becomes 200%.

FIG. 29 illustrates another example of range mapping. It is assumed thatthe marking processing unit 20 252 separates respective examples from“Case 1” to “Case 4” into two stages of (1) “highest level” and (2) “2ndhighest level” for easy understanding of the explanation.

According to the example “Case 1” shown in the figure, the range maxpercent is “8”, while the monitor luminance dynamic range is “800%”. Thepixel data of (1) “highest level” is mapped to such a value that theoutput luminance percentage becomes 800%. The pixel data of (2) “2ndhighest level” is mapped to such a value that the output luminancepercentage becomes 400%.

According to the example “Case 2” shown in the figure, the range maxpercent is “4”, while the monitor luminance dynamic range is 800%. Thepixel data of (1) “highest level” is mapped to such a value that theoutput luminance percentage becomes 400%. The pixel data of (2) “2ndhighest level” is mapped to such a value that the output luminancepercentage becomes 200%.

In case of this example, the dynamic range of the 10 video data extendsup to 400%. Accordingly, the maximum of the output luminance percentageis so selected as to correspond to 400% of the dynamic range of thevideo data even when the dynamic range of the monitor luminance extendsup to 800%. As a result, unnecessary brightness 15 and unnaturalness ofthe high-luminance part is reducible.

According to the example “Case 3” shown in the figure, the range maxpercent is “8”, while the monitor luminance dynamic range is 400%. Thepixel data of (1) “highest level” is mapped to such a value that theoutput luminance percentage becomes 400%. The pixel data of (2) “2ndhighest level” is mapped to such a value that the output luminancepercentage becomes 200%.

In case of this example, the dynamic range of the monitor luminanceextends up to 400%. Accordingly, the maximum of the output luminancepercentage is so selected as to correspond to 400% of the dynamic rangeof the video data even when the dynamic range of the monitor luminanceextends up to 400%. As a result, video data for display coinciding withthe dynamic range of the monitor luminance is obtainable, wherefore ablown-out state on the high-luminance side, i.e., so-called blown-outhighlights state is avoidable.

According to the example “Case 4”, the range max percent is “8”, whilethe monitor luminance dynamic range is 100%. The pixel data of (1)“highest level” is mapped to such a value that the output luminancepercentage becomes 100%. The pixel data of (2) “2nd highest level” ismapped to such a value that the output luminance percentage becomeslower than 100%.

Discussed next is the case of execution of the process using conversioncurve information. The range mapping processing unit 253 maps values ofrespective pixel data in a conversion target level range from V_th toV_100*N contained in transmission video data with reference to a tableof a mapping curve (mapping curve), and outputs the mapped values asoutput data. The conversion curve used in this case is a conversioncurve having a range max percent (renge_max_percent), i.e., the maximumlevel after conversion determined by using the value N and the monitorluminance dynamic range (Monitor Luminance dynamic range).

The maximum level after conversion is determined in a manner similar tothe manner when the filter information is used as discussed above (seeFIG. 29). In case of the range max percent set to “8”, and the monitorluminance dynamic range set to “800%”, for example, the maximum level tobe determined is such a value that the output luminance percentagebecomes 800%. In case of the range max percent set to “4”, and themonitor luminance dynamic range set to “800%”, for example, the maximumlevel to be determined is such a value that the output luminancepercentage becomes 400%.

As for pixel data out of the conversion target level range in thetransmission video data, values of the respective pixel data out of theconversion target level range are used as output from the range mappingprocessing unit 253 without a change, and therefore used as output fromthe HDR processing unit 205.

FIG. 30 illustrates an example (Case 5) of range mapping. According tothis example shown in the figure, the range max percent(renge_max_percent) is “4”, while the monitor luminance dynamic range(Monitor Luminance dynamic range) is 200%. In this case, the maximumlevel to be determined is such a value that the output luminancepercentage becomes 200%. According to this example, the maximum level ofthe transmission video data “960” is converted to a level “480”.

The range mapping processing unit 253 uses information on the monitorluminance dynamic range (Monitor Luminance dynamic range). When thereceiving device 200 is a set top box (STB), this monitor luminancedynamic range is allowed to be determined based on information obtainedfrom EDID on the monitor side via HDMI. The “Range_max_percent”, andrespective elements of the SEI message and the descriptor are allowed tobe shared between the set top box and the monitor when these elementsare defined in Vender Specific Info Frame. In this context, HDMI is aregistered trademark.

The operation of the receiving device 200 illustrated in FIG. 25 is nowbriefly described. The reception unit 202 receives a transport stream TStransmitted from the transmitting device 100 while carried onbroadcasting waves. This transport stream TS is supplied to the systemdecoder 203. The system decoder 203 extracts a video stream (elementarystream) from this transport stream TS. The system decoder 203 furtherextracts an HDR simple descriptor (HDR simple descriptor) or an HDR fulldescriptor (HDR full descriptor) from this transport stream TS, andtransmits the extracted descriptor to the control unit 201.

The control unit 201 recognizes whether or not tone mapping SEIinformation and HDR conversion SEI information have been inserted intothe video stream based on this descriptor. When recognizing that the SEIinformation is present, the control unit 203 is enabled to control thevideo decoder 204 such that the video decoder 204 positively acquiresthe SEI information, for example.

The video stream extracted by the system decoder 204 is supplied to thevideo decoder 204. The video decoder 204 performs a decoding process forthe video stream to generate baseband video data. The video decoder 204further extracts the SEI message inserted into this video stream, andtransmits the extracted SEI message to the control unit 201.

This SEI message contains a tone mapping information SEI message (Tonemapping information SEI message) and an HDR conversion SEI message (HDRconversion SEI message). The control unit 201 controls the decodingprocess and a display process based on the SEI information.

The video data obtained by the video decoder 204 (transmission videodata) is supplied to the HDR processing unit 205. The HDR processingunit 205 converts the high-level side level range of the transmissionvideo data such that the maximum level of the transmission video databecomes a predetermined level based on auxiliary information.

The YUV color space of the video data obtained by the HDR processingunit 206 is converted into the RGB color space by the color spaceconversion unit 206. The video data after the color space conversion issupplied to the display unit 207. The display unit 207 displays an imagecorresponding to reception video data with a luminance dynamic range ofthe transmitted video data, and further with a luminance dynamic rangein accordance with the luminance dynamic range of the monitor.

As described above, the transmitting device 100 in 30 the transmittingand receiving system 10 illustrated in FIG. 1 transmits transmissionvideo data obtained by applying a gamma curve to input video data havinga level range from 0% to 100%*N, together with transmission of auxiliaryinformation used for converting high-luminance levels on the receivingside. Accordingly, the receiving side is capable of convertinghigh-luminance levels of the transmission video data based on thisauxiliary information, for example, wherefore the receiving side iscapable of realizing display with an appropriate luminance dynamicrange.

Moreover, the transmitting and receiving system 10 illustrated in FIG. 1inserts, into a layer of a transport stream TS transmitted from thetransmitting device 100 to the receiving device 200, identificationinformation indicating that auxiliary information has been inserted intoa layer of a video stream. Accordingly, insertion of the auxiliaryinformation into the video stream is recognizable without the necessityof decoding the video stream, wherefore appropriate extraction of theauxiliary information from the video stream is realizable.

2. Modified Example

[Application to MPEG-DASH Base Stream Distribution System]

Discussed in the foregoing embodiment has been a container constitutedby a transport stream (MPEG-2 TS). However, the present technology issimilarly applicable to a system configured to realize distribution to areceiving terminal by using a network such as the Internet. In case ofdistribution via the Internet, MP4 or other format containers are oftenused for distribution.

FIG. 31 illustrates a configuration example of a stream distributionsystem 30. This stream distribution system 30 is a MPEG-DASH base streamdistribution system. According to the configuration of the streamdistribution system 30, N pieces of IPTV clients 33-1, 33-2, and up to33-N are connected with a DASH segment streamer 31 and a DASH MPD server32 via CDN (Content Delivery Network) 34.

The DASH segment streamer 31 generates DASH specification streamsegments (hereinafter referred to as “DASH segments”) based on mediadata of predetermined content (such as video data, audio data, andsubtitle data), and transmits the segments in response to an HTTPrequest from an IPTV client. The DASH segment streamer 31 may be aserver dedicated for streaming, or a server functioning as a web (Web)server as well.

The DASH segment streamer 31 further transmits segments of apredetermined stream to the IPTV clients 33 as a request source via theCDN 34 in response to a request for the segments of the correspondingstream transmitted from the IPTV clients 33 (33-1, 33-2, and up to 33-N)via a CDN 14. In this case, the IPTV clients 33 select and request astream having an optimum rate in accordance with the state of thenetwork environment where each client is present, with reference to avalue of a rate described in an MPD (Media Presentation Description)file.

The DASH MPD server 32 is a server which generates an MPD file used foracquiring DASH segments generated by the DASH segment streamer 31. TheMPD file is generated based on content metadata received from a contentmanagement server (not shown in FIG. 31), and an address (url) of thesegments generated by the DASH segment streamer 31.

According to the MPD format, respective attributes are described byutilizing elements called representations (Representations) for each ofstreams such as video streams and audio streams. For example, a rate isdescribed in an MPD file for each of representations separated incorrespondence with a plurality of video data streams having differentrates. The IPTV clients 33 are capable of selecting an optimum stream inaccordance with the respective network environments where the IPTVclients 33 are present, with reference to the values of the rates asdiscussed above.

FIGS. 32A-32D illustrate an example of relationships between respectivestructures disposed in the foregoing MPD file in a hierarchical manner.As illustrated in FIG. 32A, there exist a plurality of periods (Periods)sectioned at time intervals in a media presentation (Media Presentation)as the whole MPD file. For example, an initial period starts from 0second, while a subsequent period starts from 100 seconds.

As illustrated in FIG. 32B, a period contains a plurality ofrepresentations (Representations). The plurality of representationsinclude representation groups grouped in accordance with adaptation sets(AdaptationSets) described above, and associated with video data streamshaving different stream attributes, such as rates, and containingidentical contents.

As illustrated in FIG. 32C, a representation includes a segment info(SegmentInfo). As illustrated in FIG. 32D, this segment info includes aninitialization segment (Initialization Segment), and a plurality ofmedia segments (Media Segments) each of which describes information on acorresponding segment (Segment) divided from a period. Each of the mediasegments includes information on an address (url) and the like foractually acquiring video and audio segment data and other segment data.

A stream is freely switchable between a plurality of representationsgrouped in accordance with adaptation sets. Accordingly, a stream havingan optical rate is selectable in accordance with the network environmentwhere each of the IPTV clients is present, wherefore continuous moviedistribution is achievable.

FIG. 33A illustrates a segment structure. Segments are dividable intothree types based on differences of constituent elements. A firststructure includes a plurality of “Media Segments” for storingfragmented movie data, in addition to codec initialization information“Initialization Segment”. A second structure includes only one “MediaSegment”. A third structure includes a “Media Segment” integrated withthe codec initialization information “Initialization Segment”. FIGS. 33Band 33C illustrate examples of the data format of segments correspondingto ISOBMFF and MPEG-2TS when the structure including only one “MediaSegment is used.

When the present technology is applied to the MPEG-DASH base streamdistribution system 30, a video stream into which a tone mappinginformation SEI message (Tone mapping information SEI message) and anHDR conversion SEI message (HDR conversion SEI message) have beeninserted is disposed at the position of “Media Segment”. In addition, anHDR simple descriptor (HDR simple descriptor) or an HDR full descriptor(HDR full descriptor), and a level mapping curve descriptor (levelmapping curve descriptor) are disposed at the position of“Initialization Segment”.

FIG. 34 schematically illustrates information within a transport stream,which information corresponds to the information contained in“Initialization Segment” and the information contained in “MediaSegment” in the data format of segments corresponding to MPEG-2TS (seeFIG. 33C). As described above, the IPTV clients 33 (33-1, 33-2 and up to33-N) of the MPEG-DASH base stream distribution system 30 acquire“Initialization Segment” and “Media Segment” based on information on anaddress (url) present in the MPD file, and displays an image.

According to the stream distribution system 30 illustrated in FIG. 31,the SEI message containing gamma curve information and additionalinformation for re-mapping is similarly inserted into a layer of a videostream. Moreover, a descriptor containing identification informationindicating the presence or absence of insertion of the SEI message isinserted into a system layer (layer of container). Accordingly, the IPTVclients 33 are capable of executing processing in a similar manner tothe manner of the receiving device 200 of the transmitting and receivingsystem 10 illustrated in FIG. 1.

[Application to MMT Structure Transmission Stream]

In recent years, MMT (MPEG Media Transport) structure has beenattracting attention as a transport structure for next-generationbroadcasting. This MMT structure is chiefly characterized by coexistencewith an IP network. The present technology is also applicable to atransmitting and receiving system which handles this MMT structuretransmission stream.

FIG. 35 illustrates a configuration example of a transmitting andreceiving system 40 which handles the MMT structure transmission stream.The transmitting and receiving system 40 includes a transport packettransmitting device 300, and a transport packet receiving device 400.

The transmitting device 300 generates a transport packet having MMTstructure (see ISO/IEC CD 23008-1), i.e., a transmission streamcontaining an MMT packet, and transmits the generated transmissionstream to the receiving side via an RF transmission path or acommunication network transmission path. This transmission stream is amultiplex stream which includes a first MMT packet containing video andaudio transmission media as a payload, and a second MMT packetcontaining information concerning transmission media as a payload, in atime sharing manner and at least in a size of a fragmented packet.

The receiving device 400 receives the foregoing transmission stream fromthe transmitting side via an RF transmission path or a communicationnetwork transmission path. The receiving device 400 processestransmission media extracted from the transmission stream by using adecode time and a display time acquired based on time information, so asto display an image and output a voice.

FIG. 36 illustrates a configuration of an MMT packet in a tree form. TheMMT packet is constituted by an MMT packet header (MMT Packet Header),an MMT payload header (MMT Payload Header), and an MMT payload (MMTPayload). The MMT payload contains a message (Message), an MPU (MediaProcessing Unit), an FEC repair symbol (FEC Repair Symbol), and others.Signaling of these is executed based on a payload type (payload type)contained in the MMT payload header.

Various types of message contents are inserted into the message in atable form. The MPU is fragmented into subdivisions as MFUs (MMTFragment Units) in some cases. In this case, an MFU header (MFU Header)is added to the head of each MFU. The MMT payload contains an MPUassociated with video and audio media data, and an MPU associated withmetadata. The MMT packet containing the respective MPUs is identifiablebased on a packet ID (Packet_ID) existing in the MMT packet header.

When the present technology is applied to the transmitting and receivingsystem 40 which handles the MMT structure transmission stream, disposedas an MMT payload is such a video stream which contains insertion oftone mapping information SEI message (Tone mapping information SEImessage) and an HDR conversion SEI message (HDR conversion SEI message).Moreover, defined is such a message which has an HDR description table(HDR description table) containing contents similar to the contents ofthe foregoing HDR simple descriptor (HDR simple descriptor) or HDR fulldescriptor (HDR_full 20 descriptor) and a level mapping curve descriptor(level_mapping_curve descriptor), for example.

FIG. 37 illustrates a structure example (Syntax) of an HDR descriptionmessage (HDR description Message) having a HDR simple description table.A 16-bit field of “message_id” indicates that the structure is an HDRdescription message. An 8-bit filed of “version” indicates a version ofthis message. A 16-bit field of “length” indicates a length (size) ofthis message, showing a byte count of the subsequent part. This HDRdescription message contains an HDR simple description table (HDR simpledescription table).

FIG. 38 illustrates a structure example (Syntax) of an HDR simpledescription table. An 8-bit field of “table_id” indicates that thestructure is an HDR simple description table. An 8-bit field of“version” indicates a version of this table. In this case, “table_id”and “version” are uniquely allocated in the system. A 16-bit field of“length” indicates a whole (size) of this table. A 16-bit field of“packet_id” is identical to “packet id” contained in the MMT packetheader. This structure allows asset-level association.

A 1-bit field of “tone mapping SEI existed” is flag information whichindicates whether or not tone mapping SEI information (gamma curveinformation) is present in a video layer (layer of video stream)similarly to the HDR simple descriptor (HDR_simple_descriptor)illustrated in FIG. 18. In this case, “1” indicates that the tonemapping SEI information is present, while “0” indicates that the tonemapping SEI information is absent.

Moreover, a 1-bit field of “HDR_conversion_SEI_existed” is flaginformation which indicates whether or not HDR conversion SEIinformation (additional information) is present in the video layer(layer of video stream) similarly to the HDR simple descriptor(HDR_simple_descriptor) illustrated in FIG. 18. In this case, “1”indicates that the HDR conversion SEI information is present, while “0”indicates that the HDR conversion SEI information is absent.

FIG. 39 illustrate another structure example (Syntax) of an HDRdescription message (HDR description Message) having an HDR descriptiontable. A 16-bit field of “message_id” indicates that the structure is anHDR description message. An 8-bit filed of “version” indicates a versionof this message. A 16-bit field of “length” indicates a length (size) ofthis message, showing a byte count of the subsequent part. This HDRdescription message contains an HDR full description table (HDR fulldescription table).

FIG. 40 illustrates a structure example (Syntax) of an HDR fulldescription table. An 8-bit field of “table_id” indicates that thestructure is an HDR simple description table. An 8-bit field of“version” indicates a version of this table. In this case, ‘table_id”and “version” are uniquely allocated in the system. A 16-bit field of“length” indicates a whole (size) of this table. A 16-bit field of“packet_id” is identical to “packet_id” contained in the MMT packetheader. This structure allows asset-level association.

While not detailed herein, this HDR full description table contains“tone mapping SEI existed” and “HDR conversion SEI existed”, and furtherinformation similar to the corresponding information of the HDR fulldescriptor (HDR full descriptor) illustrated in FIG. 20.

FIG. 41 is a view illustrating a configuration example of an HDRdescription message having a level mapping curve table. A 16-bit fieldof ‘message_id” indicates that the structure is an HDR descriptionmessage. An 8-bit filed of “version” indicates a version of thismessage. A 16-bit field of “length” indicates a length (size) of thismessage, showing a byte count of the subsequent part. This HDRdescription message contains a level mapping curve table(Level_mapping_curve_table).

FIG. 42 illustrates a structure example (Syntax) of a level mappingcurve table. An 8-bit field of “table_id” indicates that the structureis a level mapping curve table. An 8-bit field of “version” indicates aversion of this table. In this case, “table_id” and “version” areuniquely allocated in the system. A 16-bit field of “length” indicates awhole (size) of this table. A 16-bit field of “packet_id” is identicalto “packet_id” contained in the MMT packet header. This structure allowsasset-level association.

While not detailed herein, information of “mapping curve table id”,“number of levels N”, “number of curve types C”, and “curve data” arecontained, similarly to the level mapping curve descriptor (levelmapping curve descriptor) illustrated in FIG. 21.

As described above, the IPTV clients 33 (33-1, 33-2 and up to 33-N) ofthe MPEG-DASH base stream distribution system 30 acquire “InitializationSegment” and “Media Segment” based on information on an address (url)present in the MPD file, and displays an image. At this time, processingusing the SEI message is achievable similarly to the receiving device200 of the transmitting and receiving system 10 illustrated in FIG. 1.

According to the transmitting and receiving system 40 illustrated inFIG. 35, the SEI message containing gamma curve information andadditional information for re-mapping is similarly inserted into thelayer of the video stream. In addition, the description table containingidentification information indicating the presence or absence ofinsertion of the SEI message is inserted into the system layer (layer ofcontainer). Accordingly, processing similar to the processing of thereceiving device 200 of the transmitting and receiving system 10illustrated in FIG. 1 is achievable by the transport packet receivingdevice 400.

The present technology may have the following configurations.

(1) A transmitting device including:

a processing unit that applies a gamma curve to input video data havinga level range from 0% to 100%*N (N: a number larger than 1) to obtaintransmission video data; and a transmission unit that transmits thetransmission video data together with auxiliary information used forconverting a high-luminance level on a receiving side.

(2) The transmitting device according to (1) noted above, wherein

the transmission unit transmits a container in a predetermined formatthat contains a video stream obtained by encoding the transmission videodata, and an

auxiliary information insertion unit that inserts the auxiliaryinformation into a layer of the video stream and/or a layer of thecontainer is provided.

(3) The transmitting device according to (2) noted above, including anidentification information insertion unit that inserts, into the layerof the container, identification information that indicates that theauxiliary information has been inserted into the layer of the videostream.

(4) The transmitting device according to any one of (1) through (3)noted above, wherein the processing unit further executes a process forconverting a level of output video data obtained by applying the gammacurve to the input video data, which level corresponds to a level of theinput video data in a range from 100% to 100%*N, into a levelcorresponding to 100% of the input video data so as to obtain thetransmission video data.

(5) The transmitting device according to (4) noted above, wherein theauxiliary information contains information on a filter applied to pixeldata of the transmission video data at a level corresponding to 100% ofthe input video data.

(6) The transmitting device according to claim any one of (1) through(3) noted above, wherein the processing unit further executes a processfor converting a level of output video data obtained by applying thegamma curve to the input video data, which level corresponds to a levelof the input video data in a range from a threshold equal to or lowerthan a level corresponding to 100% to 100%*N, into a level in a rangefrom the threshold to a level corresponding to 100% of the input videodata so as to obtain the transmission video data.

(7) The transmitting device according to (6) noted above, wherein theauxiliary information contains information on a filter applied to pixeldata of the transmission video data in a range from the threshold to alevel corresponding to 100% of the input video data.

(8) The transmitting device according to (6) noted above, wherein theauxiliary information contains information on a conversion curve appliedto pixel data of the transmission video data in a range from thethreshold to a level corresponding to 100% of the input video data.

(9) The transmitting device according to any one of (1) through (3)noted above, wherein the processing unit uses output video data as thetransmission video data without a change, which output video data isobtained by applying the gamma curve to the input video data.

(10) The transmitting device according to (9) noted above, wherein theauxiliary information contains information on a conversion curve appliedto a high-level side of the transmission video data.

(11) A transmitting method including: a processing step that applies agamma curve to input video data having a level range from 0% to 100%*N(N: a number larger than 1) to obtain transmission video data; and

a transmission step that transmits the transmission 30 video datatogether with auxiliary information used for converting a high-luminancelevel on a receiving side.

(12) A receiving device including: a reception unit that receivestransmission video data obtained by applying a gamma curve to inputvideo data having a level range from 0% to 100%*N (N: a number largerthan 1); and

a processing unit that converts a high-level side level range of thetransmission video data such that a maximum level becomes apredetermined level based on auxiliary information received togetherwith the transmission video data.

(13) The receiving device according to (12) noted above, wherein thepredetermined level is determined based on information on the N andinformation on a luminance dynamic range of a monitor contained in theauxiliary information.

(14) The receiving device according to (12) or (13) noted above, wherein

the transmission video data is video data obtained by further executinga process for converting a level of output video data obtained byapplying the gamma curve to the input video data, which levelcorresponds to a level of the input video data in a range from 100% to100%*N, into a level corresponding to 100% of the input video data, andthe processing unit converts levels of respective pixel datacorresponding to 100% of the input video data into levels in a rangefrom a level corresponding to 100% of the input video data to thepredetermined level by applying a filter specified in filter informationcontained in the auxiliary information.

(15) The receiving device according to (12) or (13) noted above, wherein

the transmission video data is video data obtained by further executinga process for converting a level of output video data obtained byapplying the gamma curve to the input video data, which levelcorresponds to a level of the input video data in a range from athreshold equal to or lower than a level corresponding to 100% to100%*N, into a level in a range from the threshold to a levelcorresponding to 100% of the input video data, and the processing unitconverts levels of respective pixel data of the transmission video datain a range from the threshold to a level corresponding to 100% of theinput video data into levels in a range from the threshold to thepredetermined level by applying a filter specified in filter informationcontained in the auxiliary information.

(16) The receiving device according to (12) or (13) noted above, wherein

the transmission video data is video data obtained by further executinga process for converting a level of output video data obtained byapplying the gamma curve to the input video data, which levelcorresponds to a level of the input video data in a range from athreshold equal to or lower than a level corresponding to 100% to100%*N, into a level in a range from the threshold to a levelcorresponding to 100% of the input video data, and

the processing unit converts levels of respective pixel data of thetransmission video data in a range from the threshold to a levelcorresponding to 100% of the input video data into levels in a rangefrom the threshold to the predetermined level by applying conversioncurve information contained in the auxiliary information.

(17) The receiving device according to (12) or (13) noted above, wherein

the transmission video data is output video data without a change, whichoutput video data is obtained by applying the gamma curve to the inputvideo data, and the processing unit converts levels of respective pixeldata of the transmission video data in a range from 10 a threshold equalto or lower than a level corresponding 100% of the input video data to alevel corresponding to 100%*N of the input video data into levels in arange from the threshold to the predetermined level corresponding to L%*100 (L: a number equal to or smaller than N) of the input video databy applying conversion curve information contained in the auxiliaryinformation.

(18) A receiving method including: a reception step that receivestransmission video data obtained by applying a gamma curve to inputvideo 20 data having a level range from 0% to 100%*N (N: a number largerthan 1); and

a processing step that converts a high-level side level range of thetransmission video data such that a maximum level becomes apredetermined level based on auxiliary information received togetherwith the transmission video data.

The present technology is chiefly characterized in that transmissionvideo data obtained by applying a gamma 30 curve to input video datawith HDR is transmitted together with auxiliary information (filterinformation and conversion curve information) used for converting ahigh-luminance level on the receiving side so as to realize display withan appropriate luminance dynamic range on the receiving side (see FIG.10).

REFERENCE SIGNS LIST

-   10 Transmitting and receiving system-   30 Stream distribution system-   31 DASH segment streamer-   32 DASH MPD server-   33-1 to 33-N IPTV client-   34 CDN-   40 Transmitting and receiving system-   100 Transmitting device-   101 Control unit-   102 Camera-   103 Color space conversion unit-   104 Gamma processing unit-   105 Video encoder-   106 System encoder-   107 Transmission unit-   200 Receiving device-   201 Control unit-   202 Reception unit-   203 System decoder-   204 Video decoder-   205 HDR processing unit-   206 Color space conversion unit-   207 Display unit-   251 Clipping processing unit-   252 Marking processing unit-   253 Range mapping processing unit-   300 Transport packet transmitting device-   400 Transport packet receiving device

The invention claimed is:
 1. A display unit comprising: a memory; andcircuitry coupled to the memory and configured to receive transmissionvideo data obtained by applying a gamma curve to input video data havingfirst luminance value range from a low luminance value to a first highluminance value, the transmission video data having a second luminancevalue range from the low luminance value to a second high luminancevalue having a smaller value than the first high luminance value; andconvert luminance values of the transmission video data having thesecond high luminance value to luminance values greater than the secondhigh luminance value and smaller than or equal to a third high luminancevalue in order to increase a maximum luminance value of the transmissionvideo data based on auxiliary information received together with thetransmission video data, without converting any luminance values of thetransmission video data smaller than the second high luminance value,wherein the transmission video data is video data obtained by executinga process for converting luminance values of output video data obtainedby applying the gamma curve to the input video data the luminance valuesof the output video data falling into a range between the second highluminance value and the first high luminance value, the convertedluminance values having the second high luminance value.
 2. The displayunit of claim 1, wherein the circuitry converts the luminance values ofthe transmission video data having the second high luminance value byapplying a filter specified in filter information contained in theauxiliary information.
 3. The display unit of claim 1, wherein themaximum luminance value is determined based on a luminance dynamic rangeof a monitor contained in the auxiliary information.
 4. The display unitof claim 1, wherein the circuitry is further configured to receive thetransmission video data from a transport stream; execute a decodingprocess of the transmission video data, including extracting asupplemental enhancement information (SEI) message inserted into thetransport stream; convert the first luminance value range responsive tothe auxiliary information; and convert a first color space of thetransmission video data to second color space.
 5. The display unit ofclaim 4, wherein the first color space in a YUV color space and thesecond color space is an RGB color space.
 6. The display unit of claim4, wherein the circuitry is further configured to determine whether tonemapping SEI information and HDR conversion SEI information have beeninserted into the transmission video stream.
 7. The display unit ofclaim 4, wherein the circuitry is further configured to extract an SEImessage inserted into the transmission video data, and to determinewhether tone mapping SEI information and HDR conversion SEI informationhave been inserted into the transmission video data.
 8. A display unitcomprising: a memory; and circuitry coupled to the memory and configuredto receive transmission video data obtained by applying a gamma curve toinput video data having first luminance value range from a low luminancevalue to a first high luminance value, the transmission video datahaving a second luminance value range from the low luminance value to asecond high luminance value having a smaller value than the first highluminance value; and convert luminance values of the transmission videodata in a range between a threshold luminance value and the second highluminance value to luminance values greater than the threshold luminancevalue and smaller than or equal to a third high luminance value greaterthan the second high luminance value in order to increase a maximumluminance value of the transmission video data based on auxiliaryinformation received together with the transmission video data, withoutconverting any luminance values of the transmission video data smallerthan the threshold luminance value.
 9. The display unit of claim 8,wherein the transmission video data is video data obtained by executinga process for converting luminance values of output video data obtainedby applying the gamma curve to the input video data, the luminancevalues of the output video data falling into a range between the firsthigh luminance value and the threshold luminance value lower than thesecond high luminance value, the converted luminance values falling intoa range between the threshold luminance value and the second highluminance value, and the circuitry converts luminance values of thetransmission video data in the range between the threshold luminancevalue and the second high luminance value into luminance values in arange between the threshold luminance value and the third high luminancevalue by applying a filter specified in filter information contained inthe auxiliary information.
 10. The display unit of claim 8, wherein thetransmission video data is video data obtained by executing a processfor converting luminance values of output video data obtained byapplying the gamma curve to the input video data, the luminance valuesof the output video data falling into a range between the first highluminance value and the threshold luminance value lower than the secondhigh luminance value, the converted luminance values falling into arange between the threshold luminance value and the second highluminance value, and the circuitry converts luminance values of thetransmission video data in the range between the threshold luminancevalue and the second high luminance value into luminance values in arange between the threshold luminance value and the third high luminancevalue by applying conversion curve information contained in theauxiliary information.